Characteristic and Universal Tensor Product Kernels

Kernel mean embeddings provide a versatile and powerful nonparametric representation of probability distributions with several fundamental applications in machine learning. Key to the success of the technique is whether the embedding is injective. This characteristic property of the underlying kernel ensures that probability distributions can be discriminated via their representations. In this paper, we consider kernels of tensor product type and various notions of characteristic property (including the one that captures joint independence of random variables) and provide a complete characterization for the corresponding embedding to be injective. This has applications, for example in independence measures such as Hilbert-Schmidt independence criterion (HSIC) to characterize the joint independence of multiple random variables.